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ABSTRACT 

This study discusses some of the central concepts, 
assumptions and methods used in the development and design of a 
Management Information and Evaluaviion System for the Peace corps in 
Colombia. Methodological problems encountered are reviewed. The model 
requires explicit project or program objectives, individual staff 
behavioral objectives, client change-objectives, specification on 
inputs, and description and verification of outputs for staff and 
client changes. Verification procedures include a work situation 
description and analysis, in-process reports and meetings, and an end 
of service report and review by field personnel %rith staff 
verification. A process and analytical outline is presented and 
discussed. An appendix is also included — Initial Analytical 
Possibilities for M. I. SB. System outputs. (FDI) 
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BTTRODUCTION 

This paper arises from a research and development project under- 
taken during 1971-72 in which the three authors helped design and imple- 
ment a Management Information and Evaluation System for Peace Corps in 
Colombia. The system attempts to incorporate basic social science prin- 
ciples while providing for sufficient reward to encourage reasonably 
rigorous application cf the system. V;e searched for those elements of 
theory and method which could be understood, appreciated and used by 
Peace Corps staff and volunteers who might be relatively unsophisticated 
in scientific method. The final product is presented here in outline 
form. A more complete description of the design and two early reports 
on its application are available in separate documents.'' 

•v7e are in the midst of a period in history when the applications 
of social science knowledge and methodology/ are becoming increasingly 
relevant to the identification and solution of societal problems. 
Applied social science is becoming more respectable as a professional 
orientation within the social science community, partially because a 
great variety of experiments - as a part of the war on poverty, urban 
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renewal, model cities programs, volunteer programs such as VISTA and 
Peace Corps, as well as other action efforts - have indicated the 
utility of social science concepts and methods in the development and 
improvement of organizational functioning- and program achievements. 

The confusion which seems to characterize many public and private 
efforts to solve social ills and realize the full development of de- 
prived segments of society is discouraging. It is difficult to justify 
continued funding or many federal, state or local programs unless we 
can present clear-cut evidence of significant success in reaching pre- 
determined goals. 

The social and behavioral sciences have had substantial difficulty 
in applying theory and method to evaluation of these programs.^ This 
is true in part because social and behavioral scientists have not often 
been trained to use their knowledge and skills in the on-going setting 
of operational programs: action program design and iniplementation is 
a frustrating enterprise, often um-ewarding in terms of peer recog- 
nition, and requires a special kind of ccirraitment that does not appear 
to be widely present among us. 

Mettiod.o_log_ical Issues 

Evaluation research in its most refined form attempts to sys- 
tematically -relate "objectives" or "goals" directly to "outputs" or 
"effects". Useful evaluation mus'. be based on methods that (in so 

P,,...?! 2^""°' £?-^Aim8Xn_Eyaluatipn Research, New Vork- 

Russell Sage Foundation, 1971, p. ix. -u-ij^. 



far as possible) yield objective and ccnprehensive evidence that sel- 
ected objectives have, or have not, led to the desired outccmes or 
effects. Likewise it is crucial to isolate the kinds of inputs or 
resources which have facilitated the achievement of (or failure to 
achieve) objectives. This requires careful recording and measuring of 
a v.'ide variety of elements within the program process if valid and 
reliable evidence is to be obtained.^ 

Two major problems a.Te often encountered in evaluation studies. 
First, program administrators are notoriously reluctant to allow 
evaluators, (particularly if they are from outside the organization and 
not subject to administrative control) access to the system. However, 
if the results can be made directl: useful for "management" purposes, 
resistance seems to mellow; hence the emphasis xn the following pages 
on "management information" in direct relationship to evaluation. 

Second, operational level organization members and their clients 
often resist evaluative interviews and hesitate to prepare reports or 
other instrument=> which supply the needed information. This suggests 
the need for a continuing "reward" or "re-enforcement" mechanism for 
any work related ^o information retrieval^ otherv/ise the quality and 
completeness of participation may be intermittent at best or generate 
inccrnplete and biased data. The system under discussion attempts to 
"build in" a reward and re-enforcment procedure that will hopefully 
increase the probability of quality and completeness of reporting. 

A broad range of other problems exist which the evaluation rc - 
searcher will undoubtedly enoount.^v b„t. for our mrpooea these two major 



3. Herbert C. Schulberg and Prank Baker. "Program Evaluation 
Modfjls and tho Tmi.l pm«nt-^Ki on uf Res<=ai-nh Findings," Ibid., pp. 72- 
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problems were most obvious.^ 

An important first requirement is the identification of how the 
results or output will be used. The system design can then be linked 
directly to these uses. In any complex organization management usually 
seeks hard data upon which decisions can be based, while also trying 
to assure task accomplishment, maintenance of employee satisfaction. 
Employees or subordinate personnel aro often concerned primarily with 
satisfaction, secondarily with task and with management decisions 
only to the degree they yield employee reward. To achieve such pur- 
poses a behavioral science based strategy is needed for development 
of an M.I,&E. system i^hich will allow for. 

(a) involvement of personnel at all ".evels in the development of 
the system, 

(b) an atmosphere in which information obtained will be accepted 
as central to the needs of management and subordinate per- 
sonnel a- Id which will lead to increased organizational pro- 
ductivity, 

(c) the development of understanding, acceptance and appreci- 
ation of the role of evaluation in the planning and manage- 
ment process; 

(d) evaluation as a continuous effort rather than a series of 
isolated events; 

(e) the introduction of a process which ultimately becomes self- 
sustaining by the program managers, staff and operational 
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level personnel; 

(f ) the measurement of success at all levels according to pre- 
determined objectives witten so as to specify expected 
performance results. 
The design of a management information and evaluation system for 
an organization must of necessity hr- vjithin the capability of the res- 
ponsible staff. Levels of management ocmpetence are by no means 
uniiorm in social and technical assistance programs, particularly in 
the Peace Corps. Because the prog-am deals with social and technoA,r.- 
logical change of great variety it is often necessary to supplement 
objective statistical data with qualitative information. 

Focusing on appropriate data parameters is facilitated by a 
series of questions which speak to the perceived needs of critical 
supporting or linked organizations. For .^eace Corps some of these 
questions include 

(a) What is the present impact of programs on the client pop- 
ulation? In the case of Peace Corps/Colombia, impact 
measurement is focused on the degree to which programs are 
meeting general Peace Corps goals, recorded objsctives as 
defined in program documents and any goals of host govern- 
ment agencies with which Peace Corps vrorks. 

(b) In what kinds of programs should Peace Corps engage in the 
immediate future? The needs of the clients served by Peace 
Corps are constantly changing in scope and depth, information 
is required by the Peace Corps bureaucracy and the U. S. 
Congreas to project needed changes well in advance of 
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specific planninc for program adjustment, 
(c) What problems or barriers might occur that could obstruct 
the development of future programs? Often an inordinate 
amount of organizational time and energy is spent dealing 
with recurring problems, without adequate information to 
comprehend and adjust to underlying causes. A systematic 
means of identifying and dealing with recurring issues is 
needed by Peace Coips management at all levels, 
(d) l^hat kind of in-country structure is needed to help the 

organization continue as a viable force in facilitating goal 
achievement? The structure needs to be flexible but with 
sufficient continuity so that programs are constantly in 
the process of pl»raaing, implementation, evaluation and 
replanring. 

Basi cjls sumptions Un^eriiiinji^a M.^ I^.^ & E-_System 

In order for a management information and evaluation system to 
become continuous certain basic assumptions must be acceptable and 
understood by all individuals responsible for some phase of the process. 
A careful orientation- educational process will usually be required to 
achieve such understanding. Some of these basic assumptions include. 

(1) Specific general goals and attainable objectives can be 
identified for programs of social and technological change. 

(2) It is possible to empirically measure whether or not the 
general goals and specific objectives established for such 
progr-ams have been achieved. 
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(3) Rewards and re-enforcement wilD be allocated so as to assiire 
achievement of personal goals, high satisfaction and continu- 
ing conmitment to organizational objectives among a high 
prcDortion of program personnel. 

(h) Behavioral objectives are a key mechanism for clarifying 

r&mtd, re-enforcement, satisfaction and ccirmitment for each 
person m the organization, if they are prepared vd.th suffic- 
ient care that results can be quantitatively or qualitatively 
measured with reasonable precision and shared with peei groups 
and managers. 

(5) On the basis of both quantitative and qualitative measures a 
degree of oojective attainment can be approximated for each 
program area, in a manner that is visible and comprehensible 
to staff, operational personnel and clients. 

(6) Measures of achievement can be analyzed and interpreted in a 
form that will be useful for decisions relating directly to 
improved ^rogranming, personnel selection, training, person- 
nel selection, training, personnel support and other manage- 
ment functions. 

Carefully refined objectives should relate directly to measures of 
verification for each operating program. If objectives cannot be obj- 
ectively verified they may be of questionable utility or relevance and 
may need re-consideration. For the system to remain valid it is impor- 
tant to avoid sudden and complete changes in the pre-determined program 
objectives. Systematic planning :s essential to realize completion of 
goals and objectives while maintaining program flexibility. 
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An explanation of each component of the System in its "ideal- 
form should serve to illuminate the incorporation of these features in 
the basic design proposed here. 

A tmAGBW INFORMATION AJJD EV'ALUATION MODEL 
The model described here is designed to fit the operational idio- 
syncrasies of Peace Corps but the concepts and approach could presuirftbly 
be adapted to fit other social and technological change programs. 
Program Goals and Qb^tt^i^jij^ 

Overall program goals are statements identifying the program area 
and long range achievement expectations. Ihe specific objectives are 
short, explicit statements describing the intended shorter ran,^e achieve- 
ments of the program. They provide sufficient specificity to establish 
the general limits of each program element. They identify the group 
or groups to be served and specify the general nature of the service 
to be provided. 

IndijnxluaJ^^PerfOTna^^ Objectivee 

The perfonnance objectives arise from the general program goals 
and specific objectives. A crucial requirement is that the expected 
behaviors be written in measurable terms and identify the tasks and 
conditions of performance to accomplish the objectives; in other words 
a statement of pre-defined performance criteria is needed. Each objective 
must identify one specific educational, developmental or service function 
to be performed, the group of groups to be served by each individual and 
the specific nature of the servioe to be provided. 
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Specific Qb.1ecti^^fl_-F>^+2:o_pij^^^ 

The ultimate goal of most social and technical change programs 
is to facilitate growth or development of a client population. If one 
can accept the thesis that human experience is measurable, then it is 
possible to prepare objectives which describe expected changes in a 
population. These client objectives should be drawn directly from the 
os^erall project objectives and from the behavioral objectives. Such 
client objectives provide direction, make more specific the long range 
goals towards which an organization is striving and represent expected 
conditions at the end of the program period, or at so- e later point in 
time depending on the nature of the program. 

In puts 

A careful documentation of all human, financial and physical re- 
sources applied to the realization of objectives should be available. 
This would include staff and skill levels of field personnel, training 
inputs, equipment, transportation, host agency inputs and all other input 
factors which will have a direct influer-e on ability to achieve objec- 
tives. 

Out£uts_-_St^;f^P^^ 

The outputs are descriptive stc^tements and statistics reflecting 
activities of all personnel engaged in the program. The statements 
should describe magnitudes of output, such as number of demonstrations 
givfn, people contacted or other quantitative indicators of activities 
and accomplishments, and include "qualitative" descriptive statements 
of activities which ere not easily qualifiable of which note factors 
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conatralning achievement of objectives. The fact that some outputs 
may go beyond or fall short of the stated objectives may indicate 
physical inability to achieve otjectives. inappropriate objectives, 
budget limits or other constraints which w^re unlmo^m during the plan- 
ning process; the output information should contain explanatory state- 
muit:, id.- ::tifying constaining or facilitating factors of this nature. 
Outputs - Clie nt Pcpula tio_nJ>ifln£P 

These statements describe changes attributable to program activity 
and/or the adoption of new behavior by che client popu3at.on. Measure- 
ment .t tlxis level becomes extremely difficult and may need to be accom- 
plished on a random sample of the client population using some of the 
techniques of diffusion research.^ 

VERIPICATION PROCEDURES 
Verifictticn of goal achievement is measured through a series of 
date collection .itages beginning prior to work initiation and ei.ding 
after departure .f Peace Corps volunteers (PCVs). Each program t:/pe 
will use information retrieval procedures adapted to the unique require- 
ments of each.^ 

.^.e-ser^fice J_ob_Des9^^^^^^ and Site /inalysis_ 

Hxese instruments are designed to solicit detailed quantitative 
and qualitative information about the current conditions at the Peace 

5. Everett Rogers with P. Ployd Schmaker. Communication of 
-%^pvatj.ons _A Cros^^^^^^^^ Pree PresV?T972^' " 

0. Pie Henry J. Jibaja. AJ^e^DortingSystem for Use in Gathering 
|^na£.onierjt_and 

thesis riontana State University, Boziian,- tot^, ilfl for a detailed 
accounting of program types in ColSmbia aAd information retSevaf^ro 
cedures used in the early data collection stages, ^^tneval pro- 
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Corps volunteer work site, they also seek information regarding poten- 
tial tasks to be performed. Vuch of the information can be collected 
from secondarjr- sources. The pre-service job description will usually 
be completed during a site visit by program staff. 

The site analysis of PCVs should be completed at the work location 
during the inital weeks of activity and seeks detailed information on 
social, cultural, technological and economic conditions. It is intended 
to provide an awareness of what conditions need to be changed as a 
prelude to establishing specific PCV behavioral objectives and any 
modifications in program objectives. The site analysis will directly 
facilitate understanding and adjustment to the work location. Since 
PCVs usually face culforal adjustment problems during their initial 
weeks, the site analysis process will serve to rapidly increase under- 
standing of the work situation, while providing inital task functions, 
which should be "rewarding- to PCVs and should facilitate staff under- 
standing of the PCV job situation. 

feSeryi^ce, .,Inf and j\e> lew . pf_ Activities 

Each PCV will generate periodic reports (roughly three month interval) 
as a basis for review and redefinition of PCV and client objectives as 
well as indicators of achievement. They will be based on systematic 
records kept by the PCVs and usually discussed at conferences with the 
staff contact, host agency supervisors and (if possible) a member of 
the client population. The reports will enable managerial staff to 
determine the operational characteristics of each program on a con- 
tinuing basis so that staff activity and program management can be 
adjusted accordingly. It provides an opportunity for field personnel 



to systematically coirinunicate with staff and local agency personnel, 
so that problems and acccmplishments can be understood and reenforce- 
ment can occur. Such interaction and infonnation exchanges should be 
"rewarding" to FCV.. staff, host agency personnel and client repre- 
sentatives; each Should feel some sense of sharing in goal achieve- 
ment and program adjustments. 

End of^ S ervice Revi ew: 

Near the completion of service at a given work site the PCV will 
systematically review his work on the site and record charges sine 
the initial site survey which can be traced directly or indirectly 
to his activities. The noted changes vdll provide a measure of the 
quantity and quality of i^act. The supervisor or program coordinator 
and evaluation officer v;ill review the final report and the field site 
to verify accuracy and completeness. 

The following diagram contains a sequential illustration of how the 
system would collect information and generate analytical reports. ^ 
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Pirst_Agpln^_at^ the System 

Implementation of the entire system at one point in time would 
obviously be difficult. Peace Corps staff in Col,>bia chose to begin 
with the periodic "volunteer planning and reporting" component (see 
block IQ on the diagram) since this would yield highly useful base- 
line data and would require involvement of essentially all staff and 
volunteers in examining the operation of existing programs. The 
"built-in reward" component for staff and FCVs is most evident in the 
planning and reporting system, since it provides specific management 
data, responds to many of the PCV concerns about inadequate attention 
from staff, helps PCV's organize their work and identifies where they 

stand With respect to Peace Corps goals as well as personal and host 

c 

country objectives. 

The initial analysis of data from the planning and reporting forms 
of nearly all volunteers (96.6;. rate of return) was based on ^'Task 
Element Descriptors," a term referring to single units of information, 
in short sentence of phrase form, describing the initiation, conduct 
or completion of a task. These "descriptors" were categorized according 
to logical groupings, as indicated in Chart I. 

iLn illustration showing a summary of tasks planned and completed 
in one three month period ig indicated in Table I, broken by program 
categories . 



o. A systematic method for helping PCV's plan their work and 
measure output has not heretofore existed in Col mbia, or most other 
Peace Corps countries; hence, the planning and reporting component 
appears to be a useful point for incorporating better planning tech- 
niques m testing the system, (other components are also in process, 
but have yet to yield published data. ) 
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CHART I 

TASK ELEMENT DESCRIPTOR CATEGORIES 



A. Contribution to client learning, 

A1 . Formal classroom instruction 

A2. Informal "extension'' instruction in a field setting 
A5. Preparation of educational or library materials 

B. Production of new information 

B1 . Studies of general value 

B2, Studies useful to groups or firms 

BJ. Studies useful to individuals only 

C. Creation of or contribution to new organizations 

D. Producing or contributing to new action programs 

E. Producing or contributing to planning of new economic or social 
development 

P. Designing or contributing to modification or reorganization of 
existing organizational structures 

G. Program maintenance tasks 

H. Non-programmed tasks 

KU Which could not have been performed by host country professionals 
m. mich could have ber^n performed by host country professionals 

!• Orientation or assistance to other PCV's 

J. Advanced preparation to increase PCV productivity 

J1. Technical training 
J2. Language training 

Host agency orientation 
K. Service to the Peace Corps program 

K1 . Assistance to staff or assumption of temporary staff position 
tud. Acquisition of supplies or equipment 

L. Other non-categor? zed work 
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Several major conclusions nay be drawn on the basis of these 
initial results 

1. Volunteers responded well to the planning and reporting system, 
in terms of returned reports and coinnitment to supporting the 
system. (96.6,'., albeit with some pressure from staff). 

2. Volunteers completed 85 percent of their planned activities. 

3. Volunteers are contributing directly and effectively to educa- 
tional programs, research, development of new organizations or 
adaptation of existing structures, planning of new development, 
generation of new action programs, and are increasing their 
own skills and abilities to conceptualize and solve problems. 
These achievements are to be expected as part of Peace Corps 
activity, but heretofore there has been little "Hard" evidence 
that presumed activities were in fact achieving objectives. 

4. Substantial staff time is required to develop and implement 
such a system if it is to fit the needs of en operational 
program. This time requirement must be counted as a short- 
term cost or investment which should lead to long term saving 
of time and resources as well as substantially greater program 
effectiveness. 

Nevertheless, it should be made clear that this initial data is by 
no means definitive and does not constitute an adequate test of the 
viability of the total system as described. There has been nearly a 
complete turnover in top level Colombian Peace Corps staff since the 
system was inaugurated, which has served to delay full in:plementation 
and testing of the 'system for all programs. However, it has proved 
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essentially workable in the cattle program where the most detailed 
early application was attempted. 

The system as designed has been widely aa.sseminated throughout 
the Peace Corps, and certain other ACTION programs but the authors 
have received little feedback to indicate the extent of application to 
date. A high level of turmoil and budget uncertainty within Peace 
Corps over the last year would lead us to assume there has been modest 
attention to the kind of systematic management information and eval- 
uation process proposed here and partially tested in Colcwbia. 
Sxmxa& ry ' ■ 

The model of a management information and evaluation system 
incorporates research design and data collection processes appropriate 
to social and technological change programs. The information generated 
could readily be used in a reasonably sophisticated research program, 
assuming that adequate rigor was used for data collection (see the 
Appendix). 

Pew federal, state, and local organizations have chosen to use 
systematic procedures in developing management irj^ormation and eval- 
uation systems. Presumably the failure to do so arises from inadequate 
knowledge of social scientific concepts among organizational staff, or 
a lack of appreciation for the utility of the ir^ormation obtained as 
an input to the management and evaluation process. The model described 
here may offer some assistance but would need substsjitial adaptation 
to assure effective applications to other socially oriented programs 
or technical assistance efforts. 

It remains to be seen, however, whether such a system can be 
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maintained without the direct guidence and involvement of professional 
soeial-behavioral scientists. Organizational managers and staff almost 
invariably tend to become overburdened with the practical day-to-day 
concerns required for task accomplishment and organizational main- 
tenance; they r.ay therefore neglect to insist upon rigorous use of 
such a system, particularly vath respect to the analytical treatment 
of information frcm which direct applications to improved organizational 
effectiveness could result. 

Regardless of its simplicity application of the model requires an 
appreciation of the difficulties in designing instruments and methods 
to obtain valid and reliable data. In many organizations and agencies 
it may be appropriate, and even crucial, to obtain the assistance of 
sophisticatea and experienced social-behavioral scientists at least 
as consultants in the design, development and monitoring of the infor- 
mation retrieval and evaluation process. 

With specific respect to the Peace Corps program in Colombia, the 
system has become sufficiently operational that outputs are already 
being used to redesign the programing of Peace Corps volunteers. 
Significant improvements have occur.'ed in volunteer selection, training, 
improvement of volunteer perfon^ance, and improvement of staff perfor- 
mance. However, it is also .lear that the application of the system 
varies substantially between individual .taff members and volunteers; 
the ccrmatment to the system depends heavily on the experience, training, 
and general orientation of the individuals involved. 

Many mar.agers feel rather strongly the "intui-oive" management, 
based on occasional written reports, and oo.asdonal contact with field 
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personnel and their clients, is less time consuming, more "personal," 
and sufficiently effective. They may be right, but how are wp to 
know unless we can ecmehow directly compare the approaches? This 
would seem to require application of an experimental model to programs 
With varying degrees of management rigor. Some efforts in this direc- 
tion have been attempted but more thorough applications of an experimental 
design are needed, possibly the "basic" and "applied" investigators 
among us can collaborate in ouoh an endeavor.^ 



9. Chapters 22-29 of the Caro book, referred to in footnote 2, 
contain several examples of experimental evaluation efforts. 
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APPENDIX 

INITIAL ANALYTICAL POSSIBILITIES PGR M. I. E. SYSTEM OUTPUTS 

1 . Analysis of volunteer impact onrthe host country based on the 
kind of PCV contributions. 

a. New learning (i.e., classes, demonstrations, talks, etc.) 

b. New information (studies, surveys, reports, analyses, etc.) 

-Studies of general value in a subject area. 

-Studies useful to a firm or group (i.e, feasibility analysis) 

-Studies useful to an individual only (i.e. soil test) 

c. New organization (i.e.. Mother's Nutrition Club, cooperative, 
etc. ) 

d. New action (i.e., construction of road or school, adoption of 
new accounting system). 

e. Planning for development (i.e., farm planning, plan for Nut- 
rition program, etc.) 

f. Designs for changed structure (i.e., production processes, 
organization of cooperatives, hospital organization, etc.) 

g. Tasks completed that could not conpetently be done by available 
Columbians (i.e., supervise school construction, supervise 
home gardens , etc . ) 

h. Orientation or learning by PCV to enable increased effectiveness, 
i • Others 

2. Complete tabulation of indicators of client change 

-Categorization by degree of change and type of change (i.e., 
behavior, attitude, group action, etc.) 

-Number and type of client involved. 

5. Tabulation and categorization of each kind of new practice or 
innovation adopted by clients of PCV's. Then- 

a. Estimate econc^ic or social value of the practice for each 
type of client or otner unit (i.e., animals vaccinated) 

b. Compute an approximate quar.titative and qualitative value of 
the change, in economic and social terms. 
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c. Compute approximate total economic and social values for all 
Peace Corps programs in Colombia, 

d. Lstimate contribution to development of Colombia for one year. 

4. Comparative analysis of cost to Colombia if Colombians were to do 
the same tasks as PCV's are now doing. 

5. Comparrlson of number and value of tasks among the different programs, 
to estimate relative value of each kind of program. 

6. Number of tasks planned but not accomplished and number of accom- 
plishments not planned, 

7. Relate number and quality of tasks accomplished to number of quarters 
of service as a PCV. 

8. Relation of individual PCV activities to 

a. Original program document (that brought PCV's to the host 
coimty) 

b. Most recent program modifications. 

9. Relative cohesiveness of programs - that is, which programs seem 
to have the least variation from the program goals in terms of 
PCV tasks undertaken. 

10. PCV effectiveness related to geographical and climatic and other 
site conditions. 



